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and technical assistance was developed in conjunction with early 
intervention/early childhood special education coordinators to ensure that 
regional and state needs were addressed and met. When possible, family 
members were also involved in the development of needs statements that guided 
and directed the outreach training and technical assistance that was 
provided. The seven training sites (Oregon, Louisiana, Washington, 
Pennsylvania, Indiana, Nevada, and Alaska) were able to choose from a menu of 
topics (e.g., curriculum-based assessment, development of functional 
Individualized Family Service Plans, involvement of families in assessment, 
naturalistic approaches to early intervention) and how they would like the 
training delivered. Participant sites were encouraged to adopt a 
" train- the- trainer " approach and were asked to send a representative to the 
initial training who could provide training to personnel located at 
individual sites. Evaluation data indicated most participants found the 
information presented useful to them and felt ready to use the information. 
The report describes the conceptual framework of the project, training 
activities, and evaluation measures and findings. (Contains 13 references.) 
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II. ABSTRACT 

Creating Systems Change to Improve Services to 
Young Children with Disabilities: An Outreach Training Project 

This project was conceived as a multi-site, multi-state outreach training project 
that proposed to use information generated from a proven model of exemplary practices 
previously funded by an EEPCD Model Demonstration Program. The outreach training 
and technical assistance was developed in conjunction with Part B and C coordinators 
and/or regional early intervention/early childhood special education coordinators to 
assure that regional/state needs were addressed and met. When possible, family 
members were also involved in the development of needs statements that guided and 
directed the outreach training and technical assistance that was provided. 

Training sites were able to choose from a menu of topics (e.g., curriculum-based 
assessment, development of functional IFSPs, involvement of families in assessment, 
naturalistic approaches to El) and how they would like the training delivered (e.g., length 
of initial training, follow-up visits, type of materials to be used). Participating sites were 
encouraged to adopt a “train-the-trainer” approach, and therefore were asked to send a 
representative to the initial training who could, in turn, provide training and assistance to 
personnel located at individual sites. 

While training content and strategies were developed to meet site/region/state 
needs, the general procedure was composed of four phases. Phase One, recruiting and 
planning, entailed extensive conversations with state coordinators to determine state 
training needs and priorities. 
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Phase Two, training, provided information, skill training and support at two levels. 
First, project staff provided participants with state-of-the-art information on the identified 
training priorities. Second, sites participating in the train-the-trainers model, received 
support and skill level development to prepare stakeholder trainers to provide training 
for designated site/region trainees on similar content. 

Phase Three, follow-up, entailed providing follow-up training to site/region 
trainees and stakeholder trainers. Options for follow-up training included: (a) providing 
additional technical assistance in areas originally identified as training objectives, (b) 
providing train-the-trainer resources and support to stakeholder trainers as they 
provided training to personnel in their sites/regions, (c) providing follow-up as needed to 
develop a system for identifying future training needs, and (d) accessing future training 
resources and evaluating training outcomes in order to maintain state-of-the-art change 
in the delivery of services. 

Phase Four, summative evaluation and dissemination, entailed evaluation of the 
project’s trainings and dissemination of findings. In addition to providing training and 
technical assistance, the project developed a set of training materials for three 
purposes: 1) to augment project training, 2) to distribute to stakeholder trainers, and 3) 
to share with sites/regions/states who did not participate directly in the outreach training. 
Evaluation of all project components was conducted using both formative and 
summative strategies. 
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IV. GOALS AND OBJECTIVES OF THE PROJECT 



The project contained two overarching goals. First, training would be provided to 
Stakeholder Trainers and Site/Region Trainees, which was intended to directly impact 
the quality of services to children with disabilities and their families. Second, the project 
was designed to provide continuous and long-term impact through a systems level 
change in one or more of the three training areas: 1) linking assessment, intervention 
and evaluation program components, 2) employing a naturalistic approach to 
intervention, and 3) family involvement. 

To impact the quality of services to children with disabilities and their families and 
provide long-term impact through systems level change, five objectives were identified. 

Objective 1 : Assist state, regional and program personnel including parents in 
the identification of needs and the development of a Collaborative Training Plan to 
systematically meet those needs. 

Objective 2 : Provide information, skill training and support to selected 
personnel, including administrators, parents; supervisors and lead teachers, to prepare 
them to serve as Stakeholder Trainers for other designated trainees throughout their 
states/regions — a train-the-trainers model. 

Objective 3 . Assist trainers in developing a culture at designated training sites to 
facilitate and maintain state of the art change in the delivery of services to young 
children with disabilities and their families. 

Objective 4 : Develop an associated set of training materials designed for the 
adult learner to augment their delivery of training content and technical assistance. 
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Objective 5 : To evaluate training/technical assistance outcomes on: a) 
Stakeholder Trainers who participate in direct training and who receive technical 
assistance and, b) Site/Region Trainees who receive training. 
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V. CONCEPTUAL FRAMEWORK FOR THE PROJECT 

Since the initiation of early intervention/early childhood special education 
(EI/ECSE) programs in the late 1960's to early 1970's, significant progress has been 
made in providing services to infants and young children with disabilities and their 
families (Bricker, Frontczak & McComas, 1998). EI/ECSE programs assist in offsetting 
the potentially negative impact of medical, biological, and environmental conditions 
associated with developmental disabilities and poor caregiving (Guralnick, 1997; 

Meisels & Shonkoff, 1990; Ramey & Ramey, 1992). While educational, medical and 
social service personnel have reason to be proud of the services currently delivered to 
participating children and families, challenges still remain. Even with obvious progress 
in the quality of programs, further improvement of intervention services offered to infants 
and children clearly remains an important goal (Diamond, Hestenes & O'Connor, 1994). 
The project titled Creating Systems Change to Improve Services to Young Children with 
Disabilities addressed three systems-level changes felt to be fundamental to the 
improvement of intervention services offered in EI/ECSE programs. The first systems 
change was the adoption and implementation by service delivery systems in targeted 
states of approaches to EI/ECSE that directly link assessment, intervention and 
evaluation. The sensible linking of the assessment, intervention and evaluation 
processes was felt to do much to improve the effectiveness and efficiency of current 
service (Bricker, in press (a); Bagnato, Neisworth, & Munson, 1989; Hutinger, 1988). 

A second systems change addressed by this project was the adoption and 
implementation of naturalistic approaches to intervention which embed training on IFSP 
goals and objectives into children's and families' daily activities (Bricker, Frontczak & 
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McComas, 1998; Kaiser, Yoder & Keetz, 1992; Wolery, Werts & Holcombe, 1994). This 
approach helps children develop skills and information that are functional, meaningful 
and useful which results in growth in problem solving and communication skills. In 
addition, naturalistic approaches can lend themselves well to the integration of children 
with disabilities into community-based programs (Bricker, Frontczak & McComas, 

1998). 

A third systems change addressed by this project was the adoption of specific 
strategies for the meaningful inclusion of caregivers in the assessment, IFSP 
development, intervention, and evaluation processes. The field has long given lip 
service to the involvement of families (Slentz & Bricker, 1992; Walker & Singer, 1993) 
but has done poorly in actually getting a broad range of family members involved in 
assessment, intervention and evaluation (Brown, Thurman, & Pearl, 1993). Specific and 
concrete strategies for family involvement are important systems changes that will likely 
do much to improve the quality of intervention services (Bricker, 1996). 

These three systems changes -- linking assessment, intervention and evaluation; 
adopting a naturalistic approach to intervention; and the meaningful inclusion of families 
-- provide the substance and content for this outreach training project and was derived 
in large measure from our previous work on an Early Education Program for Children 
with Disabilities (EEPCD) model program. 

In 1991 the Early Intervention Program, Center on Human Development, 
University of Oregon was awarded a EEPCD model demonstration grant. The major 
goal of the project was to examine the feasibility and effectiveness of using a naturalistic 
approach to intervention, called activity-based intervention, which links assessment, 
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intervention and evaluation, and involves families. During the first two years of the 
project, the approach was used at three community-based sites. During the final three 
years the approach was studied at seven replication classrooms. 

During Years One and Two the project was conducted at three community-based 
early intervention programs that served approximately 35 to 40 children per year 
ranging in age from 18 to 48 months who were at risk and disabled. A range of 
evaluation information was collected at these three sites and addressed three major 
targets: Description of selected features of the sample population and classroom 
environments; determination of the feasibility of using the approach with participating 
children and families; and determination of the effectiveness of using the approach on 
participating children. 

The demographic data provide evidence that the activity-based intervention 
approach was employed in the three different settings which served different 
populations of children and families. The data from this model demonstration project 
suggested that features of the activity-based approach were used frequently in the 
classrooms and that the approach is feasible to use during child-initiated, routine and 
planned activities. 

Based on standardized and criterion-referenced measures, the index of change 
and residual change scores indicated that children across sites made their greatest 
gains in targeted areas (i.e., IFSP goals) as compared to non-targeted areas. The data 
indicated that children made significant improvements from pretest to posttest on the 
criterion-referenced measure during both years. Finally, utility questionnaire data show 
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that parents and intervention staff rated the importance and utility of the activity-based 
approach high. 

Prior to approval for Year Three, the project was reviewed by an independent 
third party site team. The team concluded that the project had provided strong evidence 
of the effectiveness of the activity-based approach and should be approved for the 
replication phase of the project. 

Findings from the seven replication classrooms supported the outcomes from 
Years One and Two. Specifically, children made significant developmental progress, 
and teachers and parents found the activity-based approach feasible and useful. 

The evaluation outcomes from this demonstration project as well as the vast 
array of experiential knowledge we. have accumulated provided us an excellent base to 
offer the outreach training proposed in this outreach project. 
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VI. TRAINING MODEL, ADOPTION SITES, AND TRAINING ACTIVITIES 

A. Training Model 

The model for the project was composed of three major components: Linking 
assessment, intervention and evaluation; adopting a naturalistic approach to 
intervention that embeds training into children's daily activities; and meaningful inclusion 
of families. The three target groups of participants included: Part B and Part C 
Coordinators, Stakeholder Trainers, and Site/Region Trainees. The focus was to assist 
designated trainers in effecting systems level changes in their state, region and 
programs. When possible, a train-the-trainer model was employed in order to maximize 
the training impact. Training and follow-up support was offered to individual site trainers 
and site/region trainees to ensure necessary systems level change. 

The outreach training consisted of four general phases: 1) Recruitment and 
Planning; 2) Training for Site/Regional Trainees and Stakeholder Trainers; 3) Follow-Up 
for Site/Regional Trainees and Stakeholder Trainers; and 4) Evaluation and 
Dissemination. 

Phase One: Recruitment and Planning: The overall goal of the recruitment and 
planning phase was to assist state and regional level personnel to identify needs and to 
develop a plan for training and technical assistance to meet those needs. The intent of 
state level involvement and coordination was to facilitate systems level change in the 
quality of service delivery to young children and families. Part C and Part B 
coordinators were contacted to identify training needs and participants. Program 
directors, families, and lead teachers were included in this initial planning phase when 
suggested as appropriate by Part C and Part B personnel. Informal interviews, 
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telephone conferences, and electronic mail communications assisted the Project 
Coordinator in planning for training. Tasks completed by the Part B and Part C 
coordinators included: completion of a State Demographic Form, review of State-Wide 
Needs Assessment, selection of participating Sites/Regions and Stakeholder Trainers, 
and creation of the Collaborative Training Plan. 

State-level information provided general guidelines for planning in order to focus 
training on identified state needs. Each state’s unique characteristics for 
implementation of EI/ECSE services influenced the training content and the future 
implementation of the component model. The State Demographic Form is found in 
Appendix A. 

State-wide needs assessments available from targeted states were reviewed by 
the Project Coordinator. Review of the state-wide needs assessment and discussion 
focused on training needs with state level personnel which guided the identification and 
selection of specific training sites and when possible Stakeholder Trainers. In order for 
a train-the-trainers model to be effective in changing service delivery systems, the intent 
was for each state to identify an appropriate group of trainers, called Stakeholder 
Trainers. Stakeholder Trainers included administrators, family members and others 
who might influence future and sustained implementation of the model. 

Once Sites/Regions were selected to participate in the training and when 
possible Stakeholder Trainers identified, the Project Coordinator administered 
Site/Region Trainee Needs Assessment and/or the Stakeholder Trainer Needs 
Assessment to guide the development of the Collaborative Training Plan (CTP). 
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The CTP, which is similar to an Individualized Education Family Services 
Plan/Individualized Education Plan, was used throughout the project to monitor 
progress toward training objectives, to modify training or follow-up as needed to meet 
objectives, and to provide summative information concerning the impact of the project. 
The CTP included specific information concerning the training content, method of 
training, timelines, outcomes and training objectives/activities. A CTP form is contained 
in Appendix A. 

Phase 2: Training for Reqion/Site Trainees and Stakeholder Trainers . The 
project provided individualized training to a total of seven states. Training included two 
major focus areas: 1) Instructional content specified and guided by the Collaborative 
Training Plan, and 2) Instructional strategies for training others (i.e., Site/Region 
Trainees) on similar content. Multiple training materials including handouts, overheads, 
videotapes, and training modules were designed for the adult learner to augment direct 
delivery of training by the Project Liaison/Trainer and Stakeholder Trainers. 

Region/Site Trainees and Stakeholder Trainers could choose among three 
components of the training model: Linking Assessment, Intervention and Evaluation, 
Naturalistic Approach to Intervention, and Family Involvement. Within each core 
component, the Region/Site Trainees and Stakeholder Trainers selected from a menu 
of training options topics to be included during their training. In addition, Region/Site 
Trainees and Stakeholder Trainers selected the method of instruction (e.g., lecture 
format, discussion, cooperative learning techniques, videotape) based on materials 
developed for the project. The Stakeholder Trainers received training on skills for 
effective training of new Site/Region Trainees, in addition to topics within the 
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components. Table 1 indicates the model training components and a menu of training 
options associated with each one. 

After specific training objectives were identified and instructional components and 
individual topics selected, the project staff refined and developed training materials for 
each site. A goal of the project was to provide useful and pertinent training in the format 
selected by each site. Individualization of training will help ensure successful future 
trainings by Stakeholder Trainers to Site/Region Trainees. 

Following training, formative evaluation procedures were conducted at each site 
to determine if needs had been addressed. The CTP was reviewed in order to 
determine if original objectives had been achieved. Revision of the CTP by modifying or 
adding new objectives occurred when necessary to assist in long term implementation 
of the model. Follow-up site visits were scheduled as necessary to address modified 
objectives. States contributed additional monetary support for follow-up visits when 
federal dollars were not available and multiple site visits were necessary to fully 
implement systems level change. 

In addition, training recipients completed a Training Satisfaction Survey and Self 
Rating Questionnaire. The satisfaction survey was used by the Project Liaison/Trainer 
to plan and improve future trainings. The Self Rating Questionnaire identified the level 
of knowledge and readiness of the Region/Site Trainees and Stakeholder Trainers to 
meet their training outcomes. 

Phase Three: Follow-Up for Site/Reqion Trainees . Planning for follow-up 
activities began immediately following training of the Region/Site Trainees and 
Stakeholder Trainers. The CTP was reviewed to determine if follow-up and support was 
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Table 1. Menu of Training Options 



| , 1 Component 1 : 't 

Linking Assessment, Evaluation, and Intervention ' * 


Linked System Approach 


Categories of Assessment 
(Screening, Eligibility, Programming) 


Assessment,. Evaluation, and Programming System 


IFSP/IEP Development 


Writing quality goals and objectives 


Monitoring Child Progress 


Team Collaboration 


Involving Related Service Personnel in Assessment, Intervention, Evaluation 


Program Evaluation 


m : Component 2: 

Hr'." * . ? Naturalistic Intervention Approaches •. ■> * 


Activity Based Intervention: An introduction 


Environmental Arrangement 


Intervention Strategies 


Curricular Modification and Adaptation 


Data Collection Systems 


Embedding Naturally Occurring Antecedents and Consequences 


Home Consulting with ABI 


Collaborative Intervention Planning 


Involving Related Service Personnel in Naturalistic Intervention Approaches 


Components: . 

Family Involvement ‘ :!< • • 


Family Involvement in Administering, Scoring, and Interpreting Programmatic 

Assessments 


Communicating with Families 


Working with Diverse Family Structures 


Preparing the Family for the IFSP/IEP Process 
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Table 1, continued 



feaL- 



Component 3: 

Family Involvement, continued 



The Family’s Role on the Intervention Team 






Collaborative Intervention Planning 



Home Visiting 



Adapting the Curriculum for the Home Setting 



Involving Families in Progress Monitoring and Program Evaluation 



Service Coordination 



needed to meet identified outcomes and objectives. Follow-up activities that might 
occur following training included additional site visits, providing additional training 
materials or resources, contact and support via telephone, mail and/or e-mail. The goal 
of project staff was to utilize the most cost effective and efficient methods to provide 
follow-up as the Site/Region Trainees prepared for implementation of the training model 
and Stakeholder Trainers prepare for training of additional Site/Region Trainees. 

Formative evaluation procedures occurred after following-up activities to 
determine if needs had been addressed. The CTP was reviewed in order to determine 
if original objectives had been achieved. If follow-up included additional training, 
participants completed the Self-Rating Questionnaire and the Training Satisfaction 
Survey. 

Phase Four: Summative Evaluation and Dissemination . The Stakeholder 
Trainers and the Site/Region Trainees contributed summative evaluation information by 
documenting the level of attainment on each identified objective on their respective 
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Collaborative Training Plans. All summative evaluation procedures were managed by 
the Project Evaluator. In addition, data from the evaluation measures were analyzed in 
preparation for dissemination at state and national conferences and in journals, 
monographs and books. 

B. Adoption Sites and Training Activities 
A total of seven states participated in the project, adopting one or more of the 
components of the training model. The seven states were Alaska, Washington, Oregon, 
Nevada, Indiana, Louisiana, and Pennsylvania. The number of participating 
sites/regions varied from state to state. For example, in Alaska training occurred in only 
one region, Fairbanks. In Louisiana, six parishes participated in the outreach training. 

T raining activities for each of the states are included in Table 2. A total of 688 
participants received training in one of the model training components from outreach 
staff. Additional Site/Region Trainees received training on model components from 
identified Stakeholder Trainers in Indiana, Oregon and Pennsylvania. In addition to 
training, Stakeholder Trainers received training manuals and training tapes to aid them 
in providing training within their state. 
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VII. METHODOLOGICAL OR LOGISTICAL PROBLEMS 

The overarching goal of this project was to disseminate information and 
strategies garnered from our demonstration project to selected sites to assist them in 
making systems level changes that will improve services to young children with 
disabilities and their families. In spite of a few logistical problems, this overarching goal 
was achieved. This section will discuss the logistical problems encountered during 
implementation of the project as well as how they were resolved. 

Our first goal was to assist, state, regional and program personnel including 
parents in the identification of needs and the development of a Collaborative Training 
Plan to systematically meet those needs. Our intended procedure to address this goal 
was to utilize a top-down model to identify training needs and develop a Collaborative 
Training Plan. During Phase One of the project we engaged in dialogues between the 
project coordinator and Part C and Part B coordinators. These conversations were 
intended to help determine content and approaches that would best meet overall state 
needs. We had hoped that these conversations would identify a group of Stakeholder 
Trainers who would participate in initial training from project staff. 

With the exception of the state of Indiana, most Part B and C coordinators were 
unable to identify a coordinated group of personnel to serve as trainers for the entire 
state. Rather, they identified regional coordinators as a second point of contact who 
may be interested in training offered by this project. Although Part C and B coordinators 
were aware that training needs existed for local personnel, there was no statewide 
system to assess those needs. Part C and B coordinators were not adequately familiar 
with specific training needs of regional and local personnel to be able to address those 
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VII. METHODOLOGICAL OR LOGISTICAL PROBLEMS 

The overarching goal of this project was to disseminate information and 
strategies garnered from our demonstration project to selected sites to assist them in 
making systems level changes that will improve services to young children with 
disabilities and their families. In spite of a few logistical problems, this overarching goal 
was achieved. This section will discuss the logistical problems encountered during 
implementation of the project as well as how they were resolved. 

Our first goal was to assist, state, regional and program personnel including 
parents in the identification of needs and the development of a Collaborative Training 
Plan to systematically meet those needs. Our intended procedure to address this goal 
was to utilize a top-down model to identify training needs- and develop a Collaborative 
Training Plan. During Phase One of the project we engaged in dialogues between the 
project coordinator and Part C and Part B coordinators. These conversations were 
intended to help determine content and approaches that would best meet overall state 
needs. We had hoped that these conversations would identify a group of Stakeholder 
Trainers who would participate in initial training from project staff. 

With the exception of the state of Indiana, most Part B and C coordinators were 
unable to identify a coordinated group of personnel to serve as trainers for the entire 
state. Rather, they identified regional coordinators as a second point of contact who 
may be interested in training offered by this project. Although Part C and B coordinators 
were aware that training needs existed for local personnel, there was no statewide 
system to assess those needs. Part C and B coordinators were not adequately familiar 
with specific training needs of regional and local personnel to be able to address those 
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needs on a state-wide level. Unless the state had a system to identify training needs 
and a method to address those needs, the top-down model did not facilitate 
identification of potential training sites. In spite of this revelation, we were able to 
adequately identify sites in each state that had training needs in the areas of project 
focus. 

Our second objective was to provide information, skill training and support to 
selected personnel (including administrators, parents, supervisors, and lead teachers) 
to prepare them to serve as Stakeholder Trainers for other designated Site/Region 
Trainees -- A train-the-trainer model. By utilizing a train-the-trainer model we hoped to 
avoid a one-shot training approach. We felt that in order to have the most impact on 
services to children with disabilities and their families we would need to have a system 
to ensure on-going impact of training. The train-the-trainer approach would provide 
Stakeholder Trainers with information and back up support so they can return to their 
individual sites/regions and pass along the learned information designed to produce 
systems level changes (i.e., a train-the-trainers model). 

With the exception of Indiana and individual regions in Pennsylvania and 
Washington, sites were neither prepared nor ready to commit to identifying a core group 
of trainers to meet the requirements of this model. To address this problem Project staff 
decided to select sites based on strict criteria in order to facilitate systems-level change 
and to sustain that change. The most important selection criterion was based on the 
site’s commitment to adopt our model. Sites seeking basic information about our 
approach or undecided about the approach they would adopt were not considered for 
training. For example, if a site was not sure they wished to implement an activity-based 
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approach to early intervention, they were guided to selected resources to examine first. 
Once sites decided they would adopt our model they were considered for training. Even 
though most sites were not able to identify Stakeholder Trainers, we felt that their 
commitment to the model would likely increase their ability to change and sustain 
change. 

In summary, there were very few departures from the original planned activities 
and how the goals of the project would be implemented. These departures included 1) 
methods for identifying training sites and 2) adoption of the train-the-trainer model. We 
feel that neither of these modifications produced a significant impact on the 
achievement of our goals. Instead, our flexibility and willingness to adjust to the needs 
of the sites enhanced our ability to effect systems level change. 
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VI. EVALUATION FINDINGS 

The project employed both formative and summative evaluation measures to 
evaluate the goals of the project. The measures included 1) Collaborative Training 
Plan, 2) Self-evaluation measure, and 3) Training Satisfaction survey. There were five 
evaluation questions felt to be important to determine impact of training efforts which 
are presented in Table 3 with the corresponding evaluation measure. 



Table 3. Research Questions and Measures 



Research Question 


Measure 


1. Were the participants satisfied with the 
training content, presentation and 
presenters? 


Training Satisfaction Survey 


2. Did the participants find each training 
topic applicable to their roles and settings? 


Training Satisfaction Survey 


3. Did the training result in a change in 
perceived knowledge ievei of the 
participants? 


Self-Rating Questionnaire 


4. To what degree did participants perceive 
themselves as ready to implement the 
model? 


Self-Rating Questionnaire 


5. Were self-identified training outcomes 
met? 


Collaborative Training Plan 



This following section includes a description of each measure along with results 
obtained from each. 



O 
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Question 1: Were the participants satisfied with the training content, presentation 
and presenters? 

The Training Satisfaction Survey was used to answer this research question. 
Participants were asked to rate the training on three separate dimensions: 1) content 
(e.g. useful and functional), 2) presentation of material (e.g. organized and clear) and 3) 
presenters (e.g. knowledgeable and responsive). The rating on each of these three 
dimensions was based on a 5-point scale. A copy of the Training Satisfaction Survey is 
contained in Appendix A. Data pooled from all the trainings are presented in Table 4. 
Results indicated that the trainings provided by project staff met or exceeded their 
expectations on all three dimensions. 



Table 4. Evaluation of Training 





Mean 3 


Standard Deviation 


Content 


3.74 


.86 


Presentation 


3.92 


.88 


Presenters 


4.1 


.87 


Scale 1-5 with 5 being the highest rating 



Question 2: Did the participants find each training topic applicable to their roles 
and settings? 

In addition to ratings on the content, presentation, and presenters of the training, 
the Training Satisfaction Survey contained a 7-point scale intended to identify the utility 
of each training topic to the individual participant. Table 5 contains the results of this 
measure for each state participating in the project as well as data from all trainings. 
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Table 5. Participant Perception of Utility of Training 



State 


n! 






Ratings 0 






Mean 


Standard 

Deviation 


1 


2 


3 


4 


5 


6 


7 




Oregon 


1105 


19 


82 


5 


8 


362 


481 


148 


5.66 


1.24 


Washington 


937 


11 


73 


5 


1 


252 


413 


182 


5.65 


1.28 


Louisiana 


382 


2 


1 


0 


2 


133 


182 


62 


5,81 


1.02 


Pennsylvania 


229 


5 


2 


0 


2 


45 


98 


77 


5.94 


1.16 


Nevada 


145 


2 


11 


2 


1 


30 


57 


42 


5.94 


1.07 


Indiana 


137 


0 


7 


0 


2 


47 


40 


41 


5.92 


1.12 


Alaska 


113 


1 


12 


2 


0 


31 


54 


13 


5.93 


1.03 


TOTAL 


3048 


40 


188 


14 


16 


900 


1325 


565 





a n reflects the number of topics rated within each training rather than the number of 
participants in the training. 

b Ratings Key: 1 - Has doubtful utility in my setting 

2 - Understood, but I knew already 

3 - I don’t recall this was presented 

4 - Presented, but I didn’t understand it 

5 - I can use this, but need practice 

6 - I intend to use this or gain more information 

7 - I intend to try to convince others the merits of this 



Results indicate that participants’ mean rating of the utility of the training was 
almost 6. These data indicate that most participants reported that the information 
presented in the training useful to them. Most participants across states felt ready to 
use the information or gain more information. The next highest rating (5) indicated 
participants could use the information but needed more practice suggesting the need for 
follow-up training,or technical assistance to help participants who are new to the model 
implement it with confidence. 
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Question 3: Did the training result in a change in perceived knowledge level of 
the participants? 

The Self Rating Questionnaire was intended to identify the level of knowledge 
and skill on each topic of each participant. The Self-Rating Measure was administered 
prior to training and immediately after the training. Participants were asked to rate their 
knowledge on each topic using a 5-point Likert scale. Two ratings on the Self-Rating 
Questionaire were 1) knowledge on topic prior to training, 2) knowledge of topic 
immediately after training. A copy of the Self-Rating Questionnaire can be found in 
Appendix A. 

Self-rating measures were statistically analyzed using a paired-sample t-test for 

each state. Table 6 provides statistical analysis of the Self-Rating Questionnaire. 

Significant differences between pre-training and post-training means indicate 

participants had gained knowledge on target content as a result of the training. 

Question 4: To what degree did participants perceive themselves as ready to 
implement the model? 

A third rating from the Self-Rating Questionaire was used to measure 
participants’ readiness to implement the content of each topic. A state-by-state analysis 
of these data are presented in Table 7. 
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Table 6. Comparison of pre-training and post-training means across states 



State 


Pre-Training Mean 
(SD) 


Post-Training Mean 
(SD) 


df 


t Statistic 


Oregon 


2.78 


3.96 


1125 


35.14*** 




(1.5) 


(•9) 






Louisiana 


1.82 


3.88 


445 


27.40*** 




(1.20) 


(1.59) 






Washington 


1.88 


3.69 


573 


29.46*** 




(1.19) 


(1.38) 






Pennsylvania 


2.56 


3.91 


278 


16.454*** 




(1.04) 


(1.56) 






Indiana 


2.57 


3.86 


227 


15.44*** 




(1.34) 


(1.06) 






Nevada 


2.75 


3.94 


125 


8.84*** 




(1.29) 


(1.92) 






Alaska 


2.7 


3.96 


89 


11.52*** 




(1.18) 


(.52) 







*** = significant p >.05 



Table 7. Readiness to Implement on Self-Rating Measure 



State 


Mean Rating 3 


Standard Deviation 


Oregon 


3.86 


.95 


Washington 


3.78 


.91 


Indiana 


4.0 


.79 


Louisiana 


3.7 


.8 


Nevada 


4.21 


.07 


Pennslavania 


3.84 


.85 


Alaska 


3.69 


.99 



3 Scale 1-5 with 5 being the highest rating 
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Question 5. Were self-identified training outcomes met? 

The Collaborative Training Plan (CTP) is a working document developed 
cooperatively by the Stakeholder Trainers and Project Liaison and is similar to an 
Individualized Education Plan/Individualized Family Services Plan. The CTP was used 
to provide summative information concerning the impact of the project. A sample CTP 
is contained in Appendix A. 

Administrators at each training site participated in the development of the CTP. 
The CTP directed the development of training content, tailoring the content of the 
training to the individual needs of the site. In addition, the CTP was used to develop 
long-term objectives for the training participants. That is, each site developed outcomes 
they hoped to achieve toward implementation of the model after the training was 
complete. 

At a designated time following training, CTP’s were mailed back to each site for 
rating on each individualized outcome. Sites rated their progress toward each objective 
using a goal attainment scaling (1 - most unfavorable, 2 - less than expected, 3 - 
expected level, 4 - greater than expected, 5 - most favorable). Five sites chose to rate 
themselves following training on how well they achieved their objectives. Table 8 
presents the results of these ratings. 

A low return rate for the CTP’s was disappointing since the CTP was felt to be an 
important measure of sites' progress toward self-identified goals and an indicator of 
long-term impact of training efforts-. Future training efforts should emphasize strategies 
to facilitate a more consistent use of the CTP by the training sites. 
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Table 8. Goal Attainment Scaling on CTP a 



Site 


Number of 
Objectives 


Mean Rating 


Standard Deviation 


Fairbanks, AK 


4 


3 


.81 


Bellingham. WA 


6 


3.66 


1.21 


Indianapolis, Indiana 


2 


3.5 


.7 


Oak Harbor, WA 


1 


1 


0 


Toddler Learning 
Center, WA 


1 


2 


0 


TOTAL 


14 


3.14 


1.16 



Scale 1-5 with 5 being the highest 



a 
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IX. PROJECT IMPACT 

Through the three-year funding cycle a number of products have been generated 
to support participants as they implement the model and as they train others to 
implement the model. Participants committed to training others were provided with a 
Training Manual on Linking Assessment, Intervention, and Evaluation. This manual 
contains topics to help service delivery personnel implement our model, a linked system 
approach to early intervention. Trainers also received training videotapes produced by 
the project. Other products developed include assessment forms and to facilitate 
administration of the Assessment, Evaluation, and Programming System (Bricker, 

1993). Table 9 lists the products. 
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Table 9. Products 



Product 


Description/Purpose 


Assessment Activity Plans for the 
Assessment, Evaluation, and 
Programming System (AEPS) 

• For 4 Children 
By Domain 
By Script 

• For 16 children 
By Domain 

By Script 


Detailed plans which include materials and 
procedures designed for classrooms used 
to assess groups of children in a 
naturalistic environment during planned 
activities 


Group Assessment protocols for the 
AEPS 


AEPS test protocols used to assess 
groups of children using one protocol. 

This form facilitates ease in administration 
of the AEPS to a classroom of children. 


Criteria Protocols 


AEPS test protocols which include test 
criteria next to each test item. Used to 
assist new users of the AEPS to become 
familiar with item criteria 


Criteria Help Sheets 


A supplemental document to be used with 
AEPS test protocol. Used to assist 
facilitate administration of the AEPS and 
help administrators learn test criteria 
without use of test ;manual 


Cognitive Domain Help Sheet 


A supplemental document to be used with 
AEPS test protocol to promote ease of 
scoring on concepts within the cognitive 
domain of the AEPS 


Practice Scoring Videotapes 

■ 0-3 One child, One domain 

■ 0-3 One child, multiple domains 

■ 0-3 Multiple children, multiple domains 

■ 3-6 One child, multiple domains 

■ 3-6 Multiple children, multiple domains 

■ 0-3 Social-Communication Domain 

■ 3-6 Social Communication Domain 

■ Intervention Strategies associated with 
Activity-Based Intervention 


To support training for a wide variety of 
participant skill levels 
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X. FUTURE ACTIVITIES 

Since the completion of the grant several activities have occurred. First, a new 
outreach proposal, Creating and Sustaining Change Across Diverse Early Intervention 
Settings (CASCADES), was submitted and approved for funding. The CASCADES 
Project has identified three new replication sites: West Virginia, Kansas and Maine. 
West Virginia and Kansas have adopted the Assessment, Evaluation, and Programming 
System (AEPS) as one of their statewide assessments and have identified Stakeholder 
Trainers who will provide ongoing training within their states. 

Second, additional training materials are being developed to support Stakeholder 
Trainers. For example, new training tapes are being developed to support training on 
the AEPS and Activity Based Intervention. 

Third, a website is being developed to support sites who have received training in 
one of the model training components and share information about replication sites. 

The last activity is the development of the 2 nd edition of the Assessment, 
Evaluation, and Programming System (AEPS). A number of sites from this project 
chose systems level change in the area of Linking Assessment, Evaluation, and 
Intervention and adoption of the AEPS. These sites have provided valuable feedback 
about the utility of the AEPS in their programs and this information has been utilized to 
complete revisions for a 2 nd edition. 
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XI. ASSURANCE STATEMENT 

A full copy of this final report has been sent to the ERIC/OSEP Special Project of 
the ERIC Clearinghouse on Handicapped and Gifted Children, and a copy of the title 
page and executive summary have been sent to the NEC*TAS Coordinating Office. 




41 



36 



XII. REFERENCES 

Bagnato, S., Neisworth, J. & Munson, S. (1989). Linking developmental 
assessment and early interventin curricula-based prescriptions. Rockville, MD: Aspen 
Publications. 

Bricker, D. (1996). Using assessment outcomes for intervention planning: A 
necessary relationship. In M. Branbring, H., Rauh, & A. Beelmann (Eds.), Early 
childhood intervention: Theory, evaluation, and practice. Berlin/New York, NY: Walter 
de Gruyter. 

Bricker, D., Frontczak, K.,& McComas, N. (1998). An activity-based approach to 
early intervention (2 nd ed.T Baltimore, MD: Paul H. Brookes. 

Brown, W., Thurman, S.K., & Pearl, L.F. (1993). Family-centered early 
intervention with infants and toddlers: Innovative cross-disciplining approaches. 
Baltimore, MD: Paul H. Brookes. 

Diamond, K.E., Hestenes, L.L., & O'Connor, C.E. (1994). Integrating young 
children with disabilities in preschool: Problems and promise. Young Children, 49 (2). 
68-75. 

Guralnick, M.J. (1997). The effectiveness of early intervention. Baltimore, MD: 
Paul H. Brookes. 

Hutinger, P.L. (1988). Linking screening, identification, and assessment with 
curriculum. In J.B. Jordan, J.J. Gallagher, P.L. Hutinger, & M.B. Karnes: (Eds.), Early 
childhood special education: Birth to three (pp. 29-66). Reston, VA: The Council for 
Exceptinal Children and its Division for Early Childhood. 

Kaiser, A.P., Yoder, P., & Keetz, A. (1992). Evaluating milieu therapy. In S.F. 




42 



37 

Warren & J. Reichle (Eds.), Causes and effects in communication and language 
intervention (pp. 9-47). Baltimore, MD: Brookes Publishing Co. 

Meisels, S., & Shonkoff, J. (1990). Handbook of early childhood intervention. 
Cambridge, MA: Cambridge University Press. 

Ramey, S.L., & Ramey, C.T. (1992). Early educational intervention with 
disadvantaged children: To what effect? Applied and Preventative Psychology, 1. 131- 
140. 

Slentz, K. & Bricker, D. (1992). Family-guided assessment for IFSP 
development: Jumping off the family assessment bandwagon. Journal of Early 
Intervention. 16 (1). 11-19. 

Walker, B., & Singer, G. (1993). Improving collaborative communication 
between professionals and parents. In G. Singer & L. Poutrs (Eds.), Families, disability 
and empowerment. Baltimore, MD. Paul H. Brookes. 

Wolery, M., Werts, M.G., & Holcombe, A. (1994). Current practices with young 
children who have disabilities: Placement, assessment, and instruction issues. Focus 
on Exceptional Children, 26 (61. 1-12. 



ERIC 



43 



APPENDIX A 



Evaluation Measures: 

State Demographic 
Collaborative Training Planning 
Training Satisfaction Survey 
Self-Rating Questionnaire 



Creating Systems Change To Improve Services for 
Young Children with Disabilities and Their Families 

State Demographic 



The purpose of this questionnaire is to get an overall picture of how your state 
delivers services to young children with disabilities and their families and to gain 
an understanding of your statewide policies and procedures. 



State: 

Name of State Coordinator: 

Part B Part C Other (please specify) 

Title: 

Address: 



Phone Number:. 
FAX Number:_ 
e:mail Address: 



1. Who is the lead agency responsible for providing services for: 

Birth to Three 

Three to Six Years 

2. Please describe how your state is divided in terms of providing services. 
For example, list the districts, areas, parishes or regions within your state. 




S Department of Education - OSEP/OSERS - H024D60023 

Creating Systems Change to Improve Services to Young Children with Disabilities: An Outreach Project 
Early Intervention Program - University of Oregon - Eugene, OR 97403 



27-Oct-96 

STATEDEMO.DOC 
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BEST COPY AVAILABLE 



3 . 



Please describe how children in your state are identified for services 
including the following information: 



a. Please list and describe the type of agencies (e.g., private, public state 
agency) involved in identifying children who are eligible for services. 



b. Please list the key personnel (e.g., members of the multi-disciplinary 
team, early interventionists, psychologists, speech/language specialists) 
involved in identifying children who are eligible for services. 



c. Please describe the services provided (e.g., screening, diagnostic 
evaluation, case management, program placement), by the agencies 
involved in identifying children who are eligible for services. 



4. Please describe your statewide eligibility requirements. 




S Department of Education - OSEP/OSERS - H024D60023 

Creating Systems Change to Improve Services to Young Children with Disabilities: An Outreach Project 
Early Intervention Program - University of Oregon - Eugene, OR 97403 



27-OCI-96 

STATEDEMO.DOC 
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5 . 



Please describe how IFSPs/IEPs are developed in your state including the 
following information: 



a. When is the IFSP/IEP written and by whom? 



b. Are there statewide policies and procedures directing the process and 
form of how IFSPs/IEPs are written or are these decisions made on a 
regional or district wide basis? Please describe. 



6. In your state, who is responsible for working with families to determine 
program placement for children? 



7. Please check all service delivery options offered in your state 

Center-based Home-based 

Inclusive Settings Consultative 

Self-Contained Settings Other 

8. Is there anything you would like to add about the way services are 
delivered in your state? 




S Department of Education - OSEP/OSERS - H024D60023 

Creating Systems Change to Improve Services to Young Children with Disabilities: An Outreach Project 
Early Intervention Program - University of Oregon - Eugene. OR 97403 



27-Oct-96 

STATEDEMO.DOC 
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TRAINING SATISFACTION SURVEY 




Creating Systems Change 
to improve Services to 
Young Children with Disabilities 



Linked System 



Best Practices in Family 
Involvement 



Evaluating Program 
Effectiveness 



Designing Data 
Collection Systems 



Evaluating Family 
Satisfaction 



Purposes of 
Assessment 



Summarizing Assessment 
Results 



Writing Quality Goals 
and Objectives 



Communicating with 
Families 





Much less 
than expected , 


Less than 


Met 


Greater than 


Much Greater 




.Expected 


Expectation 


■•'Expected 


. than Expected 


CONTENT .J/V 

(e.g. t useful, functional) 


1 


2 


3 


4 


5 


PRESENTATION ; 

(e.g., organized, clear). . 


1 


2 


3 


4 


5 


PRESENTERS .. V 

(e.g., knowledge, responsive) 


1 


2 


3 


4 


5 



What were the strongest aspects of the workshop? 



BEST COPY AVAILABLE 



I would improve or change the workshop by: 
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